Developing search strategies for detecting high quality reviews in a hypertext test collection

OBJECTIVE To identify search strategies for retrieving high quality review studies about etiology, prognosis, therapy, and diagnosis from World Wide Web (WWW) medical documents. DESIGN Observational study of the performance of search strategies based on terms found in high quality review articles in a collection of hypertext medical documents from the WWW. MEASUREMENTS The sensitivity and specificity of search strategies for review articles in general and with a specific focus were determined by comparison to a manual review of a collection of hypertext medical documents. RESULTS A total of 1058 hypertext medical documents from seven governmental and academic WWW sites were included in the study collection. About 16% of the documents in the collection met the criteria for high quality review documents. Search strategies for review documents were identified that had 87% sensitivity and 95% specificity. Compared to simple strategies combining the term "review" and the article focus, more complex strategies based on terms found in high quality studies were more sensitive in identifying review articles of a given focus. These more complex strategies had a sensitivity of 83% for diagnosis, 85% for therapy, 79% for prognosis, and 88% for etiology, while the simple strategies had a sensitivity of 88%, 74%, 38%, and 46%, respectively. In addition, the more complex strategies were more specific for high quality review articles on diagnosis and therapy. CONCLUSION Search strategies can be identified that enhance retrieval of review documents and review documents of specific focus from a collection of WWW hypertext medical documents.